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3ACKGROUND 


s Wortp War II drew to a close, the 
Veterans Administration—faced with 
the immense challenge of rehabilitating a re- 
turning army of men, many of whom would 
need psychiatric services—recognized the ex- 
treme shortage of professional personnel 
trained to perform these services and sub- 
sequently instituted training programs in 
pertinent professional fields to help increase 
the supply of such specialists. In 1946, the 
Veterans Administration began its training 
program in clinical psychology and simul- 
taneously sponsored a program of research 
aimed at increasing the effectiveness of the 
procedures used to select and train clinical 
psychologists. 


' Early publication of this monograph was made 
possible by a special allocation to the Ann Arbor 
VA Hospital from the Education and Research 
Budget of the VA Central Office. T’re authors 
wish to acknowledge the encouragement and assist 
ance received from Philip A. Smith, formerly of 
the Ann Arbor VA Hospital, as well as the help 
of the rest of the hospital’s psychological staff 
Special thanks are due to Antonio Monteiro for the 
many months of work which he devoted to the 
analysis of the data. The salaries of Mr. Monteiro 
and the junior author were paid in part by the Vet- 
erans Administration. Statistical analysis was car 
ried out with the help of an IBM 650 Computer, 
which was freely made available by the Statistical 
Laboratory of the University of Michigan. Office 
supplies and clerical help were provided by the 
Psychology Department of the Ann Arbor VA Hos- 
pital and the Bureau of Psychological Services of 
the University of Michigan. And finally, we wish 
to thank our 245 Ss, whose contribution of their 
time and thought made this study possible 


CORRELATES OF LATER PERFORMANCE AND 
SPECIALIZATION IN PSYCHOLOGY 


A FOLLOW-UP STUDY OF THE TRAINEES ASSESSED IN THE 
VA SELECTION RESEARCH PROJECT’ 


E. LOWELL KELLY 
University of Michigan 
AND 
LEWIS R. GOLDBERG 


Stanford University 


This extensive research program (called 
the VA Selection Research Project), was 
directed and coordinated through the Psy- 
chology Department of the University of 
Michigan and carried out on a nation-wide 
basis between 1946 and 1951. The first year 
was essentially devoted to planning and car- 
rying out three preliminary assessment pro- 
grams (in Ann Arbor, Michigan; Farming- 
dale, New York; and Asilomar, California). 
During these preliminary assessments, VA 
trainees were used as Ss in an attempt to 
develop and refine techniques which were 
to be used in the later major assessments. 

he summer of 1947 marked the first of 
these major assessment programs, this in 
Ann Arbor, Michigan. One hundred and 
thirty-seven incoming graduate students 
who had just been admitted to the VA train- 
ing program in clinical psychology came to 
\nn Arbor, approximately 24 at a time, 
from all over the United States. Each group 
was subjected to an intensive series of as- 
sessment techniques during a five-day stay 
at the assessment center. 

he assessment procedures included a 
battery of objective tests and another bat- 
tery of diagnostic procedures that were 
more clinical in nature. Such objective in- 
struments as the Strong Vocational Inter- 
est Blank, the Miller Analogies Test, the 
Minnesota Multiphasic Personality Inven- 
tory, the Guilford-Martin Battery of Per- 
sonality Inventories, the Allport-Vernon 
Scale of Values, and the Thurstone Test of 
’rimary Mental Abilities were administered 
to all of these Ss. 


: 
1 : 
=. > 


E. LOWELL KELLY 


The less-structured procedures were used 
primarily to provide assessment staff mem- 
bers with bases for making descriptive, eval- 
uative, and predictive judgments about the 
candidates. The techniques included two 
varieties of interviews, a series of situational 
tests patterned after the O.S.S. assessment 
model (O.S.S. Assessment Staff, 1948), and 
four projective tests (the Rorschach Ink- 
blot Technique, the Thematic Apperception 
Test, the Bender-Gestalt Test, and a sen- 
tence completion test). In addition, each 
trainee wrote a detailed autobiography and 
as a supplement filled out a more-structured 
Biographical Inventory. Besides these 
sources of information, each staff judge had 
access to a credential file consisting of the 
trainee’s application forms to the VA pro- 
gram and to the universities of his choice, 
any correspondence with these universities, 
his letters of recommendation, and his col 
lege transcripts. 

Thirty experienced clinicians—mostly 
clinical psychologists, but with a few psy- 
chiatrists and workers—served as 
staff judges during this assessment, and 
each made qualitative and quantitative eval- 
uations on the basis of the particular assess- 
ment techniques with which he had contact. 
Great care was taken to insure that evalua 
tions based upon one technique (or group 
of techniques) would not be “contaminated” 
by information secured from other sources 
(1.e., some other technique, an informal con- 
tact with the trainee, etc.). Finally, after 
all the individual evaluations were com 
pleted, all judges who had been concerned 
with a trainee came together and, in confer 
ence, pooled their judgments to form an 
additional composite qualitative and quanti 
tative picture of each S. 

Because all the Ss utilized in this assess 
ment had been notified of their admission to 
graduate schools and to the VA training 
program, the need was felt for an additional 
assessment to be secured under applicant 
conditions. Accordingly, in 1948 another 
nation-wide assessment program was insti- 
tuted, this one using as Ss all applicants to 
the clinical psychology programs of accred- 
ited universities throughout the country. 
These applicants were tested in special cer 


social 


LEWIS R 


Gt ILDBE R¢ 


ters set up for this purpose all over the 
United States, and the protocols of their 
tests were then forwarded to a central head- 
quarters. After all applicants were tested 
and graduate schools had made their selec- 
tions, these data were evaluated by clinical 
psychologists in a manner similar to that 
employed in the 1947 assessment. 

There were two essential differences be- 
tween the two major assessment programs : 
(a) Ss in the 1947 program were evaluated 
on the basis of techniques administered 
after admission to graduate school, while 
in 1948 all testing was carried out prior to 
such acceptance, and (b) in 1947 staff 
judges had face-to-face contact with the Ss 
while the 1948 judges worked without bene- 
fit of this interaction. For the latter reason, 
such techniques as interviews and situa- 
tional tests were not available to the 1948 
judges. 

On the other hand, many techniques were 
common to the two assessment programs. 
\pplicants in 1948 took such objective tests 
as the Miller Analogies Test, the Strong 
Vocational Interest Blank, the Guilford- 
Martin Battery of Personality Inventories, 
and the Allport-Vernon Scale of Values in 
addition to such projective instruments as 
the Rorschach Ink-blot Technique and a 
sentence completion test. Again, staff judges 
had access to the complete credentials file 
for each applicant as well as to an autobi- 
ography and a Biographical Inventory. On 
the basis of these materials, evaluative rat- 
ings were made for 98 persons who were 
accepted by university departments and the 
\ A program and for 21 others who, out of 
the 545 who were completely tested upon 
application, were refused admittance to this 
program. 

lhe second half of the project was de 
voted to (a) the development of a battery 
of criterion measures, (6) the administra- 
tion of these measures to trainees near the 
end of the four-year training program, and 

*) the analyses of the interrelationships 
among the predictor and criterion measures. 
lhe findings were reported in Kelly and 
Fiske (1951) Since the Kelly-Fiske book 
is now out of print, and in order to provide 
present readers with a review of the find- 


& 


ings of the original study, a section of the 
summary chapter is quoted in Appendix A. 
The 1957 Follow-up Study 

In their summary chapter, Kelly and 
Fiske said: 

In evaluating the findings of this project, it must 
be remembered that all of the criterion measures 
were collected relatively early in the professional 
lives of the subjects studied, and that the findings 
here reported are based on these early criteria of 
professional success. It is, of course, entirely pos- 
sible that predictors which look unpromising at this 
time may have greater validity for predicting later 
riteria of professional accomplishment and vice 
‘ersa. Although the writers have no immediate 
plans for further follow-up studies, they hope to 
© able to obtain even more meaningful criterion 
measures of the subjects after 10, 15, or 20 years 
and to analyze the validities of all predictors against 
riteria of mature professional success (p. 197) 


\ number of considerations led to the 
decision not to wait even 10 years before 
undertaking a follow-up study to obtain 
later and very ditferent indices of profes- 
sional performance: The original study had 
vradually been assimilated into the main 
stream of and decisions 
regarding the selection and training of grad 
uate students could be influenced by impli- 
cations from this study. For this reason, it 
seemed desirable to have an early follow-up 
check on the later validity of some of the 
earlier findings, especially where definitive 
data were just not available in 1951 (‘e., 
eventual academic attainment) and where 
conclusions could have been reached prema 
turely. Secondly, the rapid and marked 
changes in the nature of clinical psychology 


assessment lore, 


as a specialty leads to interest in the present 
satisfaction of Ss who originally entered the 
held when somewhat at 
variance from the present state of the pro- 
fession. In addition, the relative shortage of 


conditions were 


professional personnel over the years and 
the consequent job mobility of the original 
Ss furnishes a rich background for a study 
of specialization within the field of clinical 
psychology. And finally, our colleagues and 
many of the Ss repeatedly urged us to study 
post-PhD criteria. 

Thus, early in 1957, we decided to con- 
luct the first follow-up study of the trainees 
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originally assessed in 1947 and 1948. Hav- 
ing made this basic decision we gave consid- 
erable thought to its nature, scope, and 
methodology. For reasons sentimental as 
well as scientific, we considered inviting all 
of the original Ss to return to Ann Arbor 
tor another large-scale assessment program 
or of sending trained crews of assessors all 
over the United States to obtain perform- 
ance criteria of in their 
present work Appealing as each 
was, either program would have been ex- 
tremely expensive. But, even had unlimited 
funds been available, it is not likely that 
either of them would have been feasible. 
Most of the Ss now hold responsible posi- 
tions and are hence not able to contribute 
as much time as would have been necessary. 
l‘urthermore, because of the degree of spe- 
alization within the group, it is probable 
that many would have been resistant to the 


former trainees 


settings. 


prospect of being evaluated with respect to 
skills no longer practiced. Finally, since the 
intervening years did not yield measures 
ot diagnostic and therapeutic skills superior 

the ones employed in the original study, 

simply did not seem adviszble to try to 
evaluate professional competence again at 
this time. For these reasons, it was decided 
that this first follow-up would be a modest 
one, utilizing new criteria obtainable from 
readily available sources 


Question Nairi 


One available source, the American Psy 
chological Association Directory, contained 
some information for those original Ss who 
APA members. 


tional informati n 


had become Certain addi- 
vas obtainable from the 
departments of psychology in the universi- 
ties in which they had begun their graduate 
training. Most promising, however, was the 
potential utility of a questionnaire. The 
original Ss were a highly selected group, 
capable of intelligent cooperation in a mail 
follow-up and probably sufficiently inter- 
ested to provide much of the same informa- 
tion that an interview would reveal. There 
seemed to be little value in asking for self 
evaluations of present professional compe 
tence, but we felt certain that’the Ss could, 
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and would, tell us about their professional 
work history, their work preferences as of 
today, their preferences for future work, 
and also report present satisfaction with 
their earlier professional choices. For sev- 
eral obvious reasons, both ethical and tech- 
nical, we promised not to secure evaluations 
from their present employers or supervisors. 


Administration of the Follow-up 
Ouestionnaire 


In 1957, we began a follow-up study of 
the 245 men on whom assessment evalua- 
tions were completed in 1947 and 1948. In- 
formation concerning the academic achieve- 
ments of these Ss was obtained from the 
universities in which they had been enrolled, 
and each S was requested to fill out a brief 
questionnaire pertaining to his professional 


history since leaving graduate school. A 
copy of this questionnaire—designed to pro- 
vide information concerning the Ss’ aca- 
demic and occupational histories, their work 
preferences and intentions, and their satis- 
faction with their professional choices—can 
be found in Appendix B. 

Approximately 80% of the original Ss 
were members of the American Psychologi- 
cal Association in 1957, and their addresses 
were available from this source. Attempts 
vere made to locate the remaining 20% of 
the Ss through old addresses, college alumni 
associations, telephone directories from the 
major cities in the United States, etc. Let- 
ters and questionnaires were repeatedly sent 
to each of the Ss for whom addresses were 
available until they returned a completed 
questionnaire 


TABLE 1 


ACADEMIC PROGRESS OF MALI 


ASSESSEES BY 1957 


1947 Assessment: Number assessed = 128 
1948 Assessment: Number assessed = 120 
Total assessed = 248 


Of these 248 persons: 


3 died during the interval 1947-1957. 


5 decided not to enter graduate school in psychology 
21 were not admitted to the VA program in 1948 (‘“‘Rejects’ 


Thus leaving 219 living ex-trainees. Of these 219 persons 


154 (70°%) achieved a PhD degree in psychology. 
2 (18% 


»>) achieved an MD degree. 


Of the 21 ‘Rejects’: 


13 (62°%) achieved a PhD degree in psychology 


2 (10°%) achieved an MA degree in psychology. 


1 ( 5°%) achieved an MD degree. 


2 (10°%) achieved an MA degree in other fields. 


3 (14°%) never entered graduate school. 


In summary, of the 245 living assessees: 


167 (68°;) achieved a PhD degree in psychology 


) achieved an MA degree in psychology. 
»7) are still graduate students in psychology 


b 


~) achieved PhD or MA degrees in other fields 
“o) left graduate school without any degrees 


34 (14°7) achieved an MA degree in psychology. 
12 ( 5°%) are still graduate students in psychology. 
6 ( 2°) achieved an MD degree. 
8 ( 3°>) achieved PhD or MA degrees in other fields 
18 ( 7°) achieved no known graduate degrees. 
® 1 achieved a PhD in another field. 
> 1 achieved a PhD in English literature and then returned to psy 


hology. 
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Using such techniques, questionnaires 
were received from over 93% of the 245 
living Ss, including 95% of the Ss who had 
entered the VA training program in clini- 
cal psychology, 96% of the APA members, 
and 100% of the Ss who by 1959 had 
achieved PhD degrees in psychology ! 


Tue New CRITERIA 
Academic Attainment 


Table 1 summarizes the academic prog- 
ress of all 245 Ss by 1957. Approximately 
70% of the original Ss achieved the PhD 
degree in psychology; interestingly, this 
figure was not significantly different for a 
group of 21 Ss who were not admitted into 
the VA training program in 1948. It is of 
some interest to note that 12 Ss or 5% of 
the original group still regard themselves as 
graduate students, even though their degrees 
are considerably overdue. 


lime Required ta Achieve the PhD 


There was considerable variation in the 
length of time required to complete work 
for the PhD degree in psychology, as shown 
in Table 2. Five years was the general av- 
erage (mean, median, and mode), but a 
quarter of the Ss saw their graduate educa- 
tion prolonged for six years or more. 


TABLE 2 


NUMBER OF YEARS IN GRADUATE SCHOOL PRIOR TO 
THE PHD DEGREE IN PsYCHOLOGY 


% Cumulative 

No. Yrs. No. Ss Sample % 

3 7 4% 4°, 
+ 45 25 29 
5 61 35 o4 
6 20 11 75 
7 19 10 85 
8 6 3 8g 
9 3 ? 90 
10 3 2 9? 
11 2 1 93 
12 1 1 94 
Stull students 12 7 100 

Total 79 


LATER PERFORMANCE AND SPECIALIZATION IN PSYCHOLOG\ 


ettings 


Table 3 summarizes the major work set- 
tings in 1957 of those Ss who achieved the 
PhD in psychology. VA and academic set- 
tings accounted for over half of this sam- 
ple, with the remaining Ss fairly evenly 
divided among government agencies (non- 
\'A), clinics, and hospitals. Relatively few 
Ss worked solely in private practice or in 
business settings, although almost half of 
the group reported some private practice or 
consulting work in addition to their full- 
time positions 


TABLE 3 


1957 Work SETTINGS OF THE 167 MALE ASSESSEES 
Wao ACHIEVED THE PHD IN PsyCHOLOGY 
(Mayor Position ONLy) 


No. t 

Work setting Persons Sample 

VA installations 50 30° 
Colleges or universities 42 25 

U.S. government, state or city 

agencies or systems 22 13 
Clinics, nonhospital 19 12 
Hospitals, nen-VA 18 11 
Private practice 10 6 
Business or industrial settings 6 4 

otal 167 100°; 


Satisfaction With Choice 
Psychology 


Clinical 


lable 4 indicates the reported degree of 
satisfaction associated with the choice of a 
career in clinical psychology for each of 
several subgroups of Ss categorized by prog- 
ress toward the PhD in psychology. On the 
surface, the finding that only one-half of 
the total sample would again choose to end 
up in clinical psychology (if they had their 
lives to live over again) appears rather dis 
couraging, but any meaningful interpreta- 
tion of these findings must await a compari- 
son with other professional groups. Even 
imong those receiving the PhD, only three 
out of five said they would again specialize 
in clinical psychology. 


| 
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TABLE 4 


If I had my life to live over again (knowing what I now know), 


Academic Progress 


I would try to end up in: 


Some Other 


Clinical 
Psycholog y 


PhDs in psychology 60° 
MAs in psychology * 23 
Students in psychology 42 
MDs 
PhDs or MAs in other 

fields* 
No known degrees* 


lotal* 


* Based on incomplete returns. 


Relation of Satisfaction to Job Setting 


To what extent is satisfaction with one’s 
professional choice determined by  situa- 
tional features in one’s present job (as com 
pared to more enduring personality char- 
acteristics)? As a first approximation to 
answering this question, satisfaction ratings 
were related to present work settings for 
the 167 PhD ’s in the total sample. Table 5 


Field in 
Psychology 


Other Tota! 


| 


Law or 
Business 


Medicine: 
Psychiatry 
2% 
19 


11% 
27 
17 


100° 
100 
100 


16% 
19 
42 
100 


71 
40 


100 
100 


100°, 


presents this analysis. It would appear as 
it professional satisfaction is only slightly 


related to work setting. 
Relation of Satisfaction to Present 
Specialization 

Perhaps professional satisfaction 1s 


lated to choice of specialization within the 
broad field of clinical psychology. Table 6 


TABLE 5 


SATISFACTION WITH THE FIELD OF CLINICAL PsYCHOLOGY 
as RELATED TO 1957 WorK SETTINGS 
(PHDs ONLy) 


If I had my life to live over again (knowing what I now know), 


Work Setting 


Clinical 
Psychology 


70° 
64 


VA installations 

Colleges or universities 

U.S. government, state or city 
agencies or systems 

Clinics, nonhospital 

Hospitals, non-VA 

Private practice 

Business or industrial settings 


Yotal 


l'‘would try to end up in: 


Some Other 
Field in 


Psyche logy 


All Other 
Fields 


Medicine: Total 


Psychiatry 
10°; 
24 


6 
SATISFACTION WiTH CHOICE OF CLINICAL PsYCHOLOGY BY DEGREE ATTAINED 
| 
12 
14 14 
10 30 | 
228 50°; 11% 5% | 16% | 
| | 
: 
42 5 100 
i 2? 50 ) 14 27 100 
19 63 21 16 100 
18 44 6 39 11 100 
10 70 10 20 
6 16 30 33 100 ; 
ov 12 10 13 100 
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TABLE 6 


PROFESSIONAL SPECIALIZATION WITHIN CLINICAL PsycHOLOGY RELATED TO SATISFACTION 


At the present time, I 


rily a: 


If it were not for con- 
siderations of training, 
status, salary, etc. (i.e., 
in terms of 
work involved), | would 
prefer working as a: 

In the years 
taking everything into 
consideration, the odds 
are that I will devote 
most of my time to: 


Therapist 
consider myself prima- | Teacher 


the actual | 


ahead, 


WITH THE PROFESSION 
(PaDs In PsycHOLOGY ONLy) 


If I had my lif» to live over again (knowing 
what I now kuow), I would try to end up in: 


Some Other 
Field in 
Psychology 


All 
Other 
Fields 


Medicine: 
Psychiatry 


Clinical 
Psychology 
61% S% 
72 


45 


ue 


Researcher 

Diagnostician 

Administrator- 
Supervisor 


Nm Nw 


Therapist 
Teacher 
Researcher 
Diagnostician 
Administrator- 
Supervisor 
Therapist 
Teacher 
Researcher 
Diagnostician 
Administrator- 


Totals 


100% 
100 
100 
100 


100 


100 
100 
100 
100 


100 


100 
100 
100 
100 


Supervisor 


summarizes the findings from three of the 
questions relating to professional specializa- 
tion. In general, there appear to be three 
rather clear-cut areas of specialization for 
our sample: therapists, academicians (com- 
bining teaching and research), and admin- 
istrators; only a very small percentage of 
the Ss claimed to prefer the work of diag- 
nosis or felt they would devote most of 
their time in the years ahead to diagnostic 
work. 

It would appear as if specialization is 
only slightly related to satisfaction with 
clinical psychology; however, there were 
large and consistent differences between the 
specialties with respect to which alternative 
careers seemed most appealing. That is, 
therapists were more likely than the other 
groups to state they would enter medicine 
(psychiatry) if they could begin their lives 
anew and researchers were more likely to 


100 


be attracted by other fields in psychology 
general experimental, for instance). This 
would seem to be related to the opportunities 
to carry out their preferred activities (ther- 
apy, research, etc.) in the various profes- 
sions; the few Ss who preferred the work 
of diagnosis over that of other specialties 
were the most likely to express satisfaction 
with their professional choice as clinical 


psy chologist 


Specialization as Related to Level of 
Academic Attainment 


Table 7 summarizes the replies of all Ss 
with respect to their present, most preferred, 
and estimated future areas of specialization. 

Looking first at column one, it is obvious 
that in spite of the continuing shortage of 
psychologists, not all PhDs have found it 
possible to find a position permitting them to 


27" 12% | 
= = 23 5 27 
59 5 31 5 
46 65 11 17 | 
45 69 0 24 7 : 
33 55 15 12 18 ss : 
50 56 22 10 12 = 
7 86 0 14 
31 58 3 13 26 
50; 58 0 8 
20 70 20 0 10 ae : 
29 48 24 14 14 a 
9 56 0 33 11 ais a 
08 10 7 1s | 
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TABLE 7 


pb TO LEVEL OF ACADEMIC ATTAINMENT 


Level of Academic Attainment 


Still Degrees 
PhD in MA in Graduate in Other No 
Psychology Psychology Students| Fields Degrees 
(N = 167) (N =19) (N=10) | (V=10) | (N=11) 
At the present time, I con- | Therapist 31° 6% 20% 38% | 36% 
sider myself primarily a: Teacher 15 18 0 25 0 
Researche: 13 8 10 0 0 
Diagnostician 13 35 60 12 0 
Administrator- 
Supervisor 27 24 10 25 64 


Totals 100° 

If it were not for considera- | Therapist 27° 

tions of training, status, | Teacher 20 

salary, etc. (i.e., in terms of | Researcher 30 

the actual work involved), I | Diagnostician 4 
would prefer working as a: | Administrator- 

Supervisor 19 


Totals 


Taking everything into con- | Therapist 30° 
sideration, I will probably | Teacher 12 
devote the larger part of my | Researcher 17 
time in the years ahead to: | Diagnostician 5 
Administrator- 
Supervisor 35 
Totals 100°; 


perform the work they would most prefer. 
Thus 13% reported that they were primarily 
‘Diagnosticians” but only 4% preferred that 
type of work. Only 13% considered them- 
selves primarily “Researchers” but 30% 
would prefer it. More of the PhDs were 
doing administration than would prefer it 
and an even larger proportion anticipated it 
as their most probable future work. Inter- 
estingly enough, the proportions of those 
doing, preferring, and expecting to continue 
doing therapy were very close to each other. 

While the Ns for the non-PhD groups 
are too small to yield stable percentage 
values, certain of the findings are of inter- 
est; e.g., smaller proportions doing and pre- 
ferring therapy and larger proportions do 
ing and preferring diagnosis. 


100° 100° 100°; 
16° 30°% 33% 44° 
16 20 56 0 
16 20 0 0 
21 30 0 0 
32 0 11 56 


100° 


100°; 


| 


5 30% 
21 0 30 0 
16 10 0 0 
21 50 10 0 
37 10 10 56 
100° 100°; 100°; 100° 


Scholarly Productivity 


As a rough measure of the level of schol 
arly activity of our Ss, we searched the 
indices of Psycholagical Abstracts for the 
years 1950-1957 inclusive. Omitting un- 
published theses but including all other cita 
tions, the scholarly output of the total origi- 
nal sample and for the subsample receiving 
the PhD in psychology is shown in Table 8 


Obviously, this group of psychologists 


has not contributed much volume—at least 
as yet—to the current literature of psychol 
ogy. Approximately half of them had no 


publications at all during this period, i.e., 
they did not even publish an article based 
on their theses. A quarter more have but 
one publication during the eight-vear pe 


8 
| 
; 


TABLE 8 


SCHOLARLY PRODUCTIVITY OF PHDs AND oF TOTAL 
SAMPLE OVER EIGHT-YEAR PERIOD 


No. Publi- 


Cumu- Q Cumu- 
cations in \ lative Total lative 
Psychology PhDs % Sample % 

(N = 167) (N = 245) 
0 47% 47% 58°% 58% 
1 23 70 20 78 
2 11 $1 8 86 
3or4 10 91 7 93 
5 or 6 5 96 4 97 
7 or more 4 100 3 100 


riod, in most cases based on the PhD dis- 
sertation. While the scholarly production of 
other groups of psychologists seems to be 
roughly comparable to our sample ( Dennis, 
1954), the facts reported in Table 8 raise 
serious doubts regarding the effectiveness 
of the procedures by which this group was 
presumably selected for research motiva- 
tion and trained in research methodology. 
Intercorrelations of the New Criteria 

Although these new criteria are logically 
distinguishable, it is not likely that they are 
completely independent. The actual inter- 
correlations among five of them (for the 
1948 sample of 117 assessees) are shown 
below : 


&. 

A. Academic attainment — 38 77 79 51 

B. Scholarly productivity 16 28 05 
C. Present identification with 

clinical psychology 70 51 

APA membership 18 


E. Combined criterion: Satisfied 
PhDs vs. all others 

While all of the criterion intercorrela- 
tions are positive, none is so high as to 
suggest identity and several of them are sur- 
prisingly low. From inspection of the ma- 
trix, it is obvious that scholarly productivity 
is relatively independent of the remaining 
cluster of four correlated criteria. 


CORRELATES OF THE NEW CRITERIA 
Eventual Academic Attainment 


As earlier criteria of academic perform- 
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ance, Kelly and Fiske (1951) were forced 
to rely chiefly on ratings by university staff 
members and scores on an objective content 
examination in psychology. In spite of the 
restricted range of talent in this group of 
PhD candidates, both of these criteria of 
“caliber” were predicted fairly well by 
Miller Analogies scores and by the Strong 
Psychologist Key (correlations in the .40s 
and .50s). Since over half of the total 
sample were still working toward their 
PhDs at the time the earlier report was 
prepared, it was not possible to provide a 
definitive analysis of the correlates of even- 
tual academic attainment. Preliminary anal- 
yses reported by Kelly and Fiske (1951, 
pp. 190-192) indicated that this type of 
“over-all pass-fail” criterion was not likely 
to be as predictable as criteria reflecting 
quality of graduate academic performance. 

Now that most of the original Ss have 
either obtained the PhD or have ceased 
working toward it, we are in a position to 
examine the correlates of progress toward 
the PhD. Rather than a simple two-point 
scale of “PhD vs. no PhD,” the 1957 cri- 
terion of academic progress consists of a 
five-point scale, essentially bimodal in dis- 
tribution. On one end of the scale are those 
Ss who had completed their PhD in psy- 
chology by 1957; at the other end are those 
Ss who had left graduate school without 
having received such a degree; a middle 
group (N = 12) is composed of Ss who 
were still registered students in departments 
of psychology in 1957. Each of the two ex- 
treme groups were subdivided: the non- 
PhDs were divided into groups which left 
graduate school (typically after a year 
of schooling or less) without any degree 
in psychology and another group which 
achieved MA degrees (or the equivalent) in 
psychology before withdrawing from grad- 
uate school. The PhD group was also sub- 
divided into a “fast” group which achieved 
their degrees in three to five years after en- 
trance into the VA training program and a 
“slow” group which took six to ten years 
to complete requirements for the PhD de- 
gree. 

The resulting scale of academic attain- 
ment can be summarized as follows: 


x 
3 
4 


E. LOWELL KELLY 


1 / 2 / 3 rd 4 / 5 
No 


MA Still 6-10 yr. 3-Syr. 
degree in degreein graduate PhDsin PhDsin 
psychol- psychol- students psychol- psychol- 

ogy ogy in ogy ogy 
psychol- 
ogy 


“Predictor” variables from the 1947 as- 
sessment included the Miller Analogies Test 
(MAT), all six scales of Thurstone’s Test 
of Primary Mental Abilities (PMA), all of 
the present scales of the Strong Vocational 
Interest Blank (SVIB)—including _ the 
Kreidt Psychologist Keys — the six scales 
of the Allport-Vernon Scale of Values 
(A-V), the 13 scales of the Guilford-Martin 
Battery of Personality Inventories (G-M), 
the nine standard Minne- 


scales of the 


r r 
1947 1948 
Sample | Sample 
(N=127) | (N=117) 


Variable 


Reasoning ability PMA 


.26 

VA clinical psychologist 

SVIB .26 
Psychologist — Rev. (SVIB) 17 
MAT 24 .08 
Physician SVIB .22 .03 
Reported frequency of 

expressing emotions (B1) 02 
Rated social adjustment 

(1947) 
Social extraversion (G-M) .20 — .07 
Rated intelligence (1947) .20 
Rated over-all suitability for 

clinical psychology (1947) .20 
Chemist (SVIB) .20 «2 
Public administrator (SVIB) .20 — .09 
Age at time of assessment 

(BI) —.19 .05 
Feel authors have better 

understanding of human 

nature than do teachers, 

ministers, lawyers, adver- 

tising men, or salesmen 

(BI) .19 13 
Reported membership — in 

well-knit gang during ado- 

lescence (BI) .19 — .07 
Rated broad (vs. narrow) 

interests (1947) .18 
Social worker (SVIB) .18 i 
CPA (SVIB) .18 — .07 
Physicist (SVIB) 41 
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TABLE 9 


CORRELATES OF ACADEMIC ATTAINMENT 


* Indicates that predictor variable was not available for the 1948 sample 
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Multiphasic Personality Inventory 
(MMPI), 11 of the final pooled ratings 


from the 1947 assessment (1947), and 86 


items from the Biographical Inventory 
(BI). The Pearson product-moment cor- 


relates of academic atttainment from these 
measures are summarized in Table 9, which 
lists the variables showing correlations above 
the .05 level of statistical significance for 
either the 1947 or 1948 samples together 
with the corresponding values for the other 
sample.* 


* Correlations of .17 or higher are significant at 
the .05 level (.23 or higher at the .01 level) for the 
1947 sample (N 127); correlations of .18 or 
higher are significant at the .05 level (.24 or higher 
at the .01 level) for the 1948 sample (N = 117). 


r r 


1947 | 1948 
Sample | Sample 
(N=127) | (N=117) 


Variable 


Banker (SVIB) —.17 | —.07 
Depression (MMPI) —.17 


Perceived social adjustment 

immediately precollege 

(BI) 17 — .03 
Reports associates perceive S 

to be careless about de- 


tails (BI) ae .00 
City school superintendent 

(SVIB) — .04 —.24 
Reported degree of childhood 

happiness (BI) — .24 
Number of offices held in 

high school (BI) .09 .23 
Father's educational level 

(BI — .02 — .23 
Mother's educational level 

(BI) —-.03 | —.22 
S's marital status—single vs. 


married (BI) — .06 .22 
Reported severity of inferi- 
ority feelings in childhood 


(BI) .07 .19 
Urban vs. rural childhood | 
residence (BI) — .04 .19 


Reported frequency of de- 


bilitating periods of dis- 


couragement (BI) — .13 
Father's occupational status 

(BI) .07 —.19 
Cultural and intellectual level 

ot childhood home (BI) aa —.18 


As will be noted, only 23 of nearly 200 
potential predictor variables for the 1947 
sample (and only 11 of the variables for the 
1948 sample) showed significant correla- 
tions with this new criterion of academic 
attaimment. And, even in these few in- 
stances, the resulting correlations were ex- 
tremely low—in no case accounting for as 
much as 10% of the criterion variance. 
More importantly, however, there were no 
predictor variables which showed signifi- 
cant replication in both of the samples! 


Scholarly Productivity 


As was reported earlier, the publication 
record of the two samples of former VA 
trainees does not appear to have been a dis- 
tinguished one. For this reason it seems 
all the more important to be able to locate 
potential predictors of later productivity. 
Table 10 lists the variables for the 1947 
sample which correlate significantly with 
this skewed (and rather crude) index of 
scholarly output, the number of citations in 
Psychological Abstracts during the period 
1950-1957. 

Some 33 variables correlate with pro- 
ductivity for the 1947 sample, but again 
none of the correlations account for as 
much as 10% of the criterion variance. 
And, none of these variables was replicated 
for the 1948 sample! 


APA Membership 


Approximately 80° of the original Ss 
considered their affiliation with the profes- 
sion of psychology strong enough to have 
become members of the American Psycho- 
logical Association by 1957. In an effort to 
better understand the characteristics of 
APA members (as opposed to former grad- 
uate students who drifted out of the pro- 
fession of psychology), all of the variables 
were related to 1957 APA membership 
The criterion utilized for this purpose was 
simply a two-point scale: Ss listed in the 
1957 edition of the APA Directory vs. Ss 
not so listed. The correlates of APA afflia- 
tion are shown in Table 11. 

Only 13 of the variables for the 1947 
sample and only 6 of the variables for the 


replicated for the 1947 
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CORRELATES OF SCHOLARLY PRODUCTIVITY * 


Variable 


Psychologist—Rev. (SVIB) 
Banker (SVIB) 
time S desires to devote 
to research (B1) 

Othce manager (SVIB) 

Mathematician (SVIB) 

Architect (SVIB) 

Physicist (SVIB) 

Mortician (SVIB) 

Parents’ religion — Protestant 
or Catholic vs. Jewish, 
mixed, or none (BI 

Feel authors have better 
understanding of human 
nature than do teachers, 
lawyers, ministers, adver- 
tising men, or salesmen 
(BI) 

VA clinical psychologist 
(SVIB 

Chemist (SVIB) 

rheoretical values (A-\ 

Religious values (A-V 

Rated broad (vs. narrow) 
interests (1947) 

Artist (SVIB) 

Policeman (SVIB 

Reported amount of cheating 
on exams in college (BI) 

Aesthetic values (A-\ 

Social values (A-V) 

Reported strictness of reli- 
gious training (BI 

Veterinarian (SVIB) 

First decided to become a 
psychologist early vs. late 
in life (BI) 

MAT 

Physician (SVIB 

Reported amount of guilt 
reactions to masturbation 
(BI) 

Reported regularity of church 
attendance (BI 

Political values (A-V) 

Rated intelligence (1947 

Clinical psychologist (SVIB 

Engineer (SVIB 

\ge at time of assessment 
(BI) 

Number of offices held in 
college (BI) 


1950-1957 


* Additional correlates fo 


r 
1948 

Sample» 

(N =127) | (N=117) 


.00 


.03 


* Criterion: Number of citations in Psychological Abstracts 


> 1948 sample which were not 


* are not listed in this table. 


3 
—.29 | 
28 | —.02 
—.27 02 
06 
24 .03 
m4 
—.24 —.10 
= 
24 05 
06 
.22 
21 05 
21 OF 
21 
21 .02 
2 01 
21 
.20 «a5 
— .20 - 06 
12 15 
02 
—.18 | —.17 
47 14 
17 
17) | —.04 3 
17) 
| = 
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TABLE 11 


CorRELATES OF APA AFFILIATION 


r 
1948 
Sample 


| r 


1947 


Variable Sample 


\(N=127) | (V=116) 


Reported membership in well- | 
knit gang during adoles- 
cence (BI) oad 

Age at time of assessment 
(BI) 

MAT 

Reasoning ability (PMA) 21 

Pharmacist (SVIB) 

Stated degree of preference | 
for social activities—as 
opposed to solitude (BI) 

Sales manager (SVIB) 

Urban vs. rural childhood 
residence (BI) 

Reported frequency of ex- 
pressing emotions 

Degree of participation in 
athletics in college (BI) 

Blames father (rather than 
mother) for disagreements 
between parents (BI) 

Numerical ability (PMA) 

VA clinical psychologist 
(SVIB) 

Reported severity of inferi- | 
ority feelings in childhood 
(BI) 

Feel authors have better | 
understanding of human 
nature than do teachers, 
lawyers, ministers, adver- 
tising men, or salesmen | 
(BI) 

Number of offices held in high | 
school (BI) 

Reported frequency of de- , 
bilitating periods of dis- | 
couragement (BI) 

Economic status of childhood | 
home (BI) 

Number of 
moved 
(BI) 

Age at time of first part-time 
job (Bl) 


times family 
during childhood 


1948 sample correlated significantly with 
APA membership and none of these were 
replicated between the two samples. 
Present Identification With Clinical 
Psychology 


Since about 30° of the psychologists who 


originally entered clinical psychology as 
graduate students were no longer working 
as clinicians in 1957 and no longer consid- 
ered themselves clinical psychologists, an 
attempt was made to uncover correlates of 
eventual specialization in clinical psychol- 
ogy. This criterion, like the preceding one, 
assumes no particular degree of academic 
attainment; it lumps as “nonclinicians” Ss 
who left the field of psychology entirely, 
along with PhD psychologists who are now 
identified with other areas within the pro- 
fession. Conversely, the group of “clini- 
cians” includes Ss from all levels of 
academic attainment. The correlates of 
professional identification as a clinical psy- 
chologist are summarized in Table 12. 
Again, as with the preceding analyses, 
predictor-criterion correlations are quite low 
and generally nonsignificant in the replica- 
tion sample. On the other hand, replicated 
correlates appear for the first time, but the 
correlations are so low (accounting for 
about 4% of the criterion variance) that 
their predictive value is negligible. More 
importantly, however, they have emerged 
as two out of hundreds of possible repli- 
cations and therefore could have occurred 
on the basis of chance alone. The fact that 
one variable (urban vs. rural childhood 
residence) yields a significant correlation in 
opposite directions for the two samples sug- 
gests the plausibility of this interpretation. 


Satisfaction With Choice of Clinical 
Psychology 


Approximately 50% of the Ss stated that 
they would enter clinical psychology again 
if they had their lives to live over, with the 
remainder choosing other more preferred 
occupational choices (see Table 4). In an 
attempt to identify earlier correlates of later 
professional satisfaction, all of the variables 
were correlated with this dichotomous cri- 
terion. Table 13 summarizes this analysis. 

Again it is obvious that the correlations 
are generally quite low, and there emerged 
no replicated correlates of this satisfaction 
criterion. Consequently, additional analyses 
of professional satisfaction were attempted. 


12 
.02 
.06 3 
.13 
— .05 
.04 
— .03 
.12 
—.17 
.14 
— .16 
| .24 
2 .24 
.20 
8 — .20 
3 
3 
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CORRELATES OF PRESENT IDENTIFICATION 


WITH CLINICAL PsYCHOLOGY 


Variable 


Social extraversion (G-M) 


Urban vs. rural childhood 


residence (BI) * 

Number of “‘psychosomatic”’ 
symptoms reported (BI) 
Preference for diagnostic in- 

terviewing—as opposed to 

therapy or research (BI) 
Age at time of first date (BI) 
Reported frequency of ex- 

pressing emotions (BI) 


Parents’ religion—Protestant | 


or Catholic vs. Jewish, 
mixed, or none (BI) 
Reasoning ability (PMA) 
VA clinical psychologist 
(SVIB) 
Personnel director (SVIB) 
Social worker (SVIB) 


Special promotion(s) in school | 


(BI)> 
Feel authors have better 
understanding of human 


nature than do teachers, | 


lawyers, ministers, adver- 
tising men, or salesmen 
(BI)> 

Hypomania (MMPI) 

Rated broad (vs. narrow) 
interests (1947) 

Rated degree of talkativeness 
(1947) 

Rated degree of emotionality 
(1947) 

MAT 

Psychologist—Rev. (SVIB) 

Public administrator (SVIB) 

Cultural and intellectual level 
of childhood home (BI) 

Economic values (A-V) 

Spatial ability (PMA) 

Depression (MMPI) 

Number of offices held in high 
school (BI) 

Social science high school 
teacher (SVIB) 

Occupational level (SVIB) 

Minister (SVIB) 


® Statistically significant correlate in opposite directions for 


the two samples. 


> Replicated correlate for 1947 and 1948 samples 


r 
1947 1948 
Sample | Sample 
(N=127) | (N=117) 
31 — .06 
— .26 
—.25 .10 
24 .00 
23 — .09 
.23 - 04 
.08 
.22 .07 
.22 .16 
21 20 
.20 20 
.20 
.20 
| 20 
19 sid 
.18 .06 
| .18 — .06 
| rj — .03 
—.17 .07 
— 
—.17 
.00 28 
.08 
— .15 —.19 
.09 .18 


SPECIALIZATION IN PSYCHOLOGY 


TABLE 13 


CORRELATES OF SATISFACTION WITH PREVIOUS 


Variable 


MAT 
Rated intelligence (1947) 


Rated broad (vs. narrow) | 


interests (1947) 

Reasoning ability (PMA) 
Rated over-all suitability for 
clinical psychology (1947) 

VA clinical psychologist 
(SVIB) 

Reported frequency of de- 
bilitating periods of dis- 
couragement (BI) 

Reported feelings regarding 
number of  siblings—too 
few to too many (BI) 

Political values (A-V) 

Depression (G-M) 

Cycloid disposition (G-M) 

Inferiority feelings (G-M) 

Nervousness (G-M) 

Cooperativeness (G-M) 

Rated social adjustment 
(1947) 

Social worker (SVIB) 

Theoretical values (A-V) 


Psychologist—Rev. (SVIB) 
Osteopath (SVIB) 
Marital status—single vs. 


married (BI) 

Age at time of first date (BI) 

Experimental psychologist 
(SVIB) 

Number of offices held in high 
school (BI) 

Aesthetic values (A-V) 

Reported difficulty in leaving 
parents during adolescence 
(BI) 

Guidance psychologist 
(SVIB) 

Degree of participation in 
athletics in high school 
(BI) 

Reported amount of guilt 
feelings in childhood (BI) 


See Appendix B). 


CHOICE OF CLINICAL PSYCHOLOGY * 


r 


1947 


Sample | 
(N=127) | (N=117) 


2 
.26 


.24 


O01 


.O1 


r 
1948 
Sample 


— .02 


.00 


® Criterion: If I had my life to live over again (knowing 
what I now know) I would try to end up in clinical psychology 
vs. all others (Question 9 of 1957 Follow-up Questionnaire 
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21 — .02 
20 08 
-.20 09 
.18 11 
.18 04 
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Satisfaction With Area of Specialization 
Within Psychology 


Perhaps satisfaction with one’s area of 
specialization within the profession is a less 
elusive criterion than general occupational 
satisfaction. To check on this, an index of 
satisfaction with one’s area of specialization 
was constructed by examining the questions 
of the follow-up questionnaire relating to 
present, most preferred, estimated- 
future work. Roughly 50% of the Ss who 
answered these questions were carrying out 
activities which they professed to prefer 
most and simultaneously estimated they 
would continue with in the years ahead. The 
other half of the sample either preferred 
other activities than those they were carry- 
ing out or estimated that in the future they 
would be unable to continue with their most- 
preferred activity, or both. As in the pre- 
ceding analyses all variables were correlated 
with this dichotomous criterion of satisfac- 
tion; the findings are summarized in Table 
14. 

\gain, correlations were low and no rep 
lications occurred. Vor this reason, still 
other analyses were attempted in an effort 
to tind consistent correlates of various as 
pects of professional satisfaction.’ In each 
instance a promising pattern of predictors 
for the 1947 sample did not replicate for 
the 1948 group! 


Academic Success Plus Satisfaction With 
Professional Choice 


Approximately 40% of the total sample 
had achieved a PhD degree in psychology 
by 1957 and simultaneously expressed satis 
faction with their professional choice (by 
stating that they would have entered the 
field of clinical psychology if they could 
live their lives over again). The other 60% 
either had not achieved the PhD degree in 
psychology or felt that they would enter 
some other field if they could live their lives 
anew. Since it was felt that such a com- 
bined index might be a conceivable criterion 
upon which to judge initial selection proce 


* The analyses of the various indices of satistfa 
ion were all carried out by Antonio Monteir 


TABLE 14 


CORRELATES OF SATISFACTION WITH PRESENT AREA 
OF SPECIALIZATION WITHIN PsyCHOLOGY * 


r r 
1947 1948 
Variable Sample | Sample 


(N =127) | (N=117) 


Public administrator (SVIB) 26 —.11 
Nervousness (G-M) 25 .06 
Masculinity-femininity (G-M) 23 .04 
Inferiority feelings (G-M) 23 .06 
Depression (G-M) 21 08 
Rated intelligence (1947) 21 
Rated over-all suitability for 

clinical psychology (1947) 21 
Reported participation in 

extracurricular activities in 

high school (BI) 21 .05 


Reported amount of cheating 

on exams in college (BI) -.21 .04 
Reported amount of sexual 

experience during adoles- 


cence (BI) 21 — .03 
Cycloid disposition (G-M) .20 — .08 
Production manager (SVIB) .19 5 
Reported satisfaction with 

high school grades (BI) .00 


Reported frequency of de- 
bilitating periods of dis- 


couragement (BI) -.18 .03 
Masculinity-femininity 
(SVIB) — .02 


Both parents present in child- 

hood home—vs. one or 

more missing (BI) sz — .05 
Father (vs. mother) reported 

as final authority in child- 


hood discipline (BI) — .07 
Special promotion(s) in 

school (BI) .00 
Urban vs. rural childhood 

residence (BI) .00 
City school superintendent 

(SVIB) .08 —.25 
Reported feeling regarding 

number of  siblings—too 

few to too meny (BI) .09 es 
Religious values (A-V) — .04 .20 
Number of times family 

moved during childhood 

(BI) 20 
Pharmacist (SVIB 04 .20 


Reported degree of ‘‘bookish- 

ness” of S during adoles- 

cence (BI) .06 19 
Reported regularity of par- 

ents’ church attendance 

(BI .18 


* Criterion: Preferred and estimated-future area of specia 
zation both identical with present area of specialization 
Questions 6, 7, and 8 of the 1957 Follow-up Questionnaire 
See Appendix B 


; 


dures (since ideally a profession might well 
strive for both “successful” and “satisfied” 
practitioners), all variables were correlated 
with this dichotomous multiple criterion. 
Table 15 summarizes these findings. 

Again the correlations are low and the 
only replicated variables are two highly 
correlated Strong scales (Social Worker 
and Personnel Director). As was pointed 
out earlier, the possibility of this replication 
emerging on the basis of chance factors 
alone must be considered. 


The Absence of Significant and Consistent 
Correlates of the New Criteria 


In spite of the socially relevant (and rea- 
sonably objective) nature of the new cri- 
teria, it is clear that only a small proportion 
of their variance is predicted by the assess- 
ment data obtained prior to the Ss entry to 
graduate schools. While the findings re- 
ported deal primarily with the more objec- 
tive assessment variables, additional analy- 
ses suggest that the pooled ratings of the 
assessment staff (based on projective and 
interview data) fare no better as predictors 
of these new criteria. At first glance, it 
seems unusually puzzling that these seven 
real-life criteria appear to be completely 
uncorrelated with nearly 200 predictor 
measures covering a wide range of psycho- 
logical domains. How could this be? Ac- 
knowledging the risks of post hoc hypothe- 
sizing, a few possible explanations merit 
discussion. 

Although many of the new criteria ap- 
pear to be objective indices of performance, 
nonetheless they may mask sizable differ- 
ences in the actual functioning of the Ss. 
For example, the standards of performance 
demanded for the PhD are probably quite 
varied across universities. Standards of ac- 
complishment in academic and practicum 
work may vary enormously in schools and 
agencies with differing goals and methods 
of training. Thus, a criterion such as attain- 
ment of a PhD—while desirably reflecting 
some uniformity in training and assessed 
competence—may in fact be remarkably 
situationally specific. Table 16 presents 
some evidence for this point of view. AIl- 
though the small size of the samples for each 
university make statistical analysis unfeas- 
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TABLE 15 
CORRELATES OF THE COMBINED CRITERIA: ACADEMIC 
Success (PHD) PLus SATISFACTION WITH THE 
CHOICE OF CLINICAL PsyCHOLOGY 


r 
1947 1948 
Sample | Sample 
(N=127)| (N=117) 


Variable 


Rated intelligence (1947) .26 
VA clinical psychologist 

(SVIB) “a0 .06 
Objectivity (G-M) ‘an 
MAT .24 — .05 
CPA (SVIB) “ae — .09 
Reasoning ability (PMA) .22 
Cooperativeness (G-M) .07 
Memory (PMA) 20 
Rated broad (vs. narrow) 

interests (1947) .20 
Social worker (SVIB) * .20 21 
Perceived social adjustment 

immediately precollege (BI) .20 «Al 
Reported frequency of de- 

bilitating periods of dis- 

couragement (BI) — .20 03 
Political values (A-V) —.19 02 
Rated social adjustment 

(1947) 19 
Personnel director (SVIB)* 19 22 
Theoretical values (A-V) .18 .07 
Social extraversion (G-M) 18 .02 
Depression (G-M) 18 
Inferiority feelings (G-M) .18 
Rated over-all suitability for 

clinical psychology (1947) 18 
Veterinarian (SVIB) 18 — .01 
Reported ability to control | 

emotions (BI) .18 
Spatial ability (PMA) ee 
Cycloid disposition (G-M) .09 
Ascendance—Submission 

(G-M) .08 
Psychologist—Rev. (SVIB) ae .04 
Number of offices held in 

college (BI) 
Marital status—single vs. 


married (BI) .07 | .29 
Reported amount of difficulty 

in leaving parents during | 

adolescence (BI) .00 


— .23 

Guidance psychologist (SVIB) .04 .20 
Reported amount of guilt 

feelings in childhood (BI) -.01 .20 
Father's occupational status 

(BI) — .02 — .20 
Age at time of first date (BI) | .03 — .20 
Social science high school 

teacher (SVIB) .04 19 
Experimental psychologist 

(SVIB) — .02 —.19 
Perceived satisfaction of 

parents regarding S’s high 

school grades (BI .09 


® Replicated relate for 1947 and 1948 samples. 
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TABLE 16 


INTERUNIVERSITY DIFFERENCES IN THE ACADEMIC 
ATTAINMENT, APA AFFILIATION, AND PROFESSIONAL 


| % Com- | fied with 

pleting | Choice of 

| PhD | Clinical 
versity | No. Ss | Members| in Psy- Psy- 

Code | Assessed | in 1957 | chology | chology 


Uni- 


50% | 28% 
75 38 
40 0 
86 59 
80 40 
67 58 
50 
0 
100 
14 
67 
44 
20 
100 
71 
67 
0 
67 
33 


A 


OSD 


Totals 
37 245 167 113 


Av. % 68% 46% 


* Includes one S who received his PhD at another university 


> Includes two Ss who received their PhDs at other universi 
ties. 


ible, it is apparent that there are considera- 
ble interuniversity differences (for example, 
University BB vs. University DD), in the 


eventual outcome of their graduate students 
on these three criteria. To the extent that 
these differences exist, no measures of the 
Ss alone could ever predict such a criteria 
effectively! To the extent that our criteria 
reflect the judgments of other persons (i.e., 
teachers and supervisors in the case of aca- 
demic attainment, editors in the case of 
scholarly productivity ), maximal prediction 
demands the assessment of both those doing 
the judging as well as those being judged. 
For a more detailed elaboration of this po- 
sition, see Stern, Stein, and Bloom (1956). 

Even criteria such as perceived satisfac- 
tion with psychology as a profession and 
satisfaction with an area of specialization 
probably depend in large part upon the 
kinds of settings in which the Ss have 
worked and upon the kinds of experiences 
which they have encountered in these set- 
tings. And again, to the extent that the 
variance in satisfaction indices are deter- 
mined by situationally specific variables, to 
this extent are they unpredictable from 
measures (no matter how inclusive and 
accurate) of the Ss alone. 

The possible situationally specific nature 
of many of the criterion measures appears 
to be the most reasonable explanation for 
the general low correlations found in this 
follow-up study. Nevertheless, there are 
other possible explanations. It is possible 
that the range of predictor variables was 
just not wide enough to include those varia- 
bles most relevant for the prediction of the 
eventual criteria; similarly, it is conceivable 
that the relevant abilities, interests, and atti- 
tudes most highly correlated with the new 
criteria are “late-flowering” and only be- 
come measurable after the exposure to cer 
tain experiences during training (or after a 
certain number of years of chronological- 
psychological maturation). 

In addition, possible explanations for the 
lack of significant correlates may lie in the 
methodology of the study. Because of the 
essentially dichotomous nature of most of 
the new criteria and because of the labor- 
saving advantages of machine analysis of 
the hundreds of variables under study, no 
attempt was made to look for nonlinear re 
lationships which may exist between predic 
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SATISFACTION OF THEIR GRADUATE STUDENTS 4 
| | 
% Satis- 
| } 75% | 
B 100 | 
40 
D 2 96 
E 100 
F | 1 100 
H 100 1 
I 100 
J 57 
K 83 
L | 78 
M | 1 80 
N | 100 1 
O 86 
67 
Q 50 
R 100 | 
S 6 50 
r | 9 78 OC 78 89 
U 3 100 100 | 33 
V 8 88 75 =| 75 
WwW 6 67 50* 50 
xX 3 100 100 100 
7 4 75 75 50 
Z 5 80 60 20 
AA 6 100 67." 67 ’ 
BB 10 60 40 0 
6 100 83 50 
DD | 13 92 69 62 4 
EE 5 60 60 40 
FF 8 50 50" 62 
GG 100 100° 25 
HH 8 88 885 38 
Il 4 100 758 75 
MM | 1 100 0 0 
NN 1 100 100 100 ; 
“Re- 
jects” 21 62 62 24 


tor and criterion variables. It is possible 
that the actual relationships between the two 
sets of variables may not be as simple as a 
straight line (i.e., perhaps Ss in the middle 
range of some variable are most apt to be 
highly productive or to attain the PhD 


while those on the extremes of the variable 


tend to leave the profession). And, the re- 
lationship between predictor and criterion 
variables might well be even more compli- 
cated, involving patterns of two or more 
variables, demonstrable only by configura- 
tional analyses. In many ways it seems more 
realistic to assume that criteria of the kind 
with which we were working should bear a 
more complex relationship with predictor 
variables than the simple straight line. 

Another methodological problem of this 
study involves the reliability of the criterion 
indices. In the case of academic attainment, 
it is fair to assume that there will be no 
marked changes in this criterion during the 
ensuing years. In the case of scholarly pro- 
ductivity, however, this is not so obvious. 
The findings of Girden and Dennis (1954, 
p. 633) that psychologists reach their maxi- 
mum productivity rates in their early thir- 
ties and that the rate of schoiarly output 
appears to be quite stable over their profes- 
sional careers, suggest that this criterion 
might also be a reasonably reliable one (the 
median age for our Ss was approximately 
35). On the other hand, the measures of 
satisfaction were obtained on the basis of 
answers to a questionnaire and it is possible 
that such answers might fluctuate as a func 
tion of present moed. 

The above discussion has been concerned 
solely with the lack of predictively useful 
correlations between the two sets of varia- 
bles, the most disconcerting finding of this 
study. Almost equally disconcerting, how 
ever, is the general finding that what appears 
to be a meaningful pattern of signifi 
cant (even though weak) predictor varia- 
bles of each criterion for the 1947 group 
does not reappear when we examine the 
1948 sample. 

Why this failure to replicate? There ap- 
pears to be at least three possible explana 
tions: chance, conditions of assessment, and 
actual differences in the two ‘samples 
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Chance. The tables in this section were 
constructed by selecting from nearly 200 
those variables which correlated signifi- 
cantly at the .05 level or better with the 
criterion. In most cases, the number of 
significant correlations was greater than 5% 
of the total. However, the predictor vari- 
ables are far from independent, and hence, 
the number of “significant” correlations to 
be expected by chance alone is an undeter- 
mined value greater than 5%. 

Conditions of assessment. The 1947 
group came to Ann Arbor for a week-long 
volunteer assessment program in the sum- 
mer of 1947 after they had been accepted 
by a graduate school and the VA training 
program. By contrast the 1948 sample were 
required to complete the assessment battery 
at designated centers in the spring of 1948, 
before they had been admitted to graduate 
school, under the impression that their test 
scores would (or at least might) be used as 
criteria of selection by the graduate school 
of their choice. Actually, this was not the 
case, since the scores were never reported 
back to the universities. Nevertheless, it is 
not unlikely that taking the tests under an 
apparent selection situation may have re- 
sulted in sufficient “faking good” of re- 
sponses to have resulted in the differences 
i correlations for the two groups. While 
this tendency —especially on the part of psy- 
chology majors—-may have influenced the 
scores of certain of the inventories, it is not 
likely that it affected performance on the 
MAT or responses to the more factual 
questions of the BI. 

Real differences in the two samples. 
Even though either of the above explana- 
tions is sufficient to explain the differences 
in results for the two samples, we should 
not overlook the possibility that the groups 
differed in one or more critical ways which 
would lead to different real patterns of re- 
lationships between the predictor and cri- 
terion variables. Thus far we have not 
identified any such suppressor or modulating 
variables, but this seems to be a lead worth 
further investigation. It should be remem- 
bered that most of the 1947 group had com- 
pleted college before entering the military 
service and many of them had service duties 
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closely related to, if not actually in, clinical 
psychology. By contrast, the 1948 Ss, while 
also mostly veterans, returned from the 
service to complete their AB or BS degrees. 
A large portion of them were seniors during 
the 1947-1948 school year, at which time 
the VA program represented a very attrac- 
tive opportunity for graduate and profes- 
sional training—especially when compared 
to alternatives then available. 

Whatever the correct explanation of the 
failure to replicate, it is clear that, at the 
present time, we simply do not have ade- 
quate bases for predicting these real-life 
criteria of professional status. If it is as- 
sumed that the lack of consistent correla- 
tions is the result of such good selection 
practice that all accepted applicants are 
equally likely to succeed on any of several 
criteria, this would be regarded as a healthy 
state of affairs. To the authors, however, 
it seems more likely that our findings reflect 
the interaction of several factors about 
which we should not be too complacent : 

1. Marked differences among universities 
with respect to: (a) caliber of students ac 
cepted for the PhD; (>) aptitudes sought in 
graduate mathematical vs 
verbal; (c) types of courses offered and 
required for the PhD; (d) relative empha 
sis in teaching and grading on memory, 
originality, creativity, rigor, etc.; (e) rela- 
tive emphasis on performance in courses, 
field work, and thesis; (f) nature of and 
standards for the PhD thesis. 


students, e.g., 


2. The absence of clear-cut and agreed 
upon performance criteria for professional 
employment and promotion. In view of the 
differences enumerated under No. 1 above, 
it is probable that “PhD in Psychology,” 
while an objective criterion, has surpris 
ingly little uniformity of meaning from uni 
versity to university and from specialty to 
specialty. Add to this the fact that we have 
no agreed-upon criteria of competence in 
teaching, diagnosis, therapy, or administra 
tion, nor for the quality of publications, and 
it is hardly surprising that the variance in 
criteria seems to be determined 


our new 


largely by situational rather than intra 


individual variables 


AND 
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PERSONALITY CORRELATES OF 
SPECIALIZATION IN PsyCHOLoGy‘* 


Although our efforts to find consistent 
correlates of performance and satisfaction 
within the field of psychology were not suc- 
cessful, the early work of Strong (1943) 
and later of Nachmann (1957) had already 
demonstrated marked differences between 
various vocational groups. In the past few 
years a number of studies have focused on 
the personality correlates associated with 
different professions, comparing groups 
whose work activities appeared quite dissim- 
ilar (i.e., accountants and creative writers 
Segal, 1953]). Since many professions, 
including psychology, are extremely broad 
in function and their members engage in 
quite different kinds of work activities, it 
appears reasonable to assume that persons 
who choose (or are chosen) to perform 
similar types of work—regardless of the 
profession with which they are identified— 
will have certain important personality char- 
acteristics in common, and thus may well 
have shared some critical developmental ex- 
periences. Conversely, within any broad 
occupational group, members who perform 
different types of work activities might be 
expected to differ from each other in a 
manner congruent with their differing intra- 
vocational preferences. To test this assump- 
tion within the framework of psychology, 
samples of Ss representing major fields of 
specialization were compared on variables 
from the objective measures common to the 
1947 


Originally 


and 1948 assessments 


five areas of specialization 


considered: therapy, diagnosis, re- 
search, teaching, and administration. How- 


ever, responses to the 1957 follow-up ques- 


were 


tionnaire revealed few Ss who intended to 
devote their professional lives to the prac- 
and who 
teachers or “pure” research- 


tice of diagnosis few Ss were 
“pure” 

than some combination of the 
was 


three 


rather 


eTs 


activities). For this reason, it 


two 


judged most practical to consider 


major groups of specialists: therapists, aca- 


' This section is based on the junior author’s doc 


| 
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demicians (representing the combined activ- 

ities of teaching and research), and admin- 
istrators. In a sense, these three subfields 
can be seen as reflecting a logical division 
among many professions—the practitioner, 
the academician, and the administrator being 
in no way unique to the field of psychology. 
For this reason, it is hoped that some of the 
correlates of specialization among psycholo- 
gists might be relevant to choice of special 

ization among other professional groups. 

In order to increase the probable homo 
geneity of the specialist criterion groups, the 
samples of each were selected from among 
the total 1947 and 1948 samples to include 
only persons who (a) reported they pre- 
ferred the work of their specialty to that 
of any other and (+) who simultaneously 
stated that they believed they would devote 
the largest proportion of their future pro- 
fessional careers to the same work. Using 
these criteria, 20 relatively “pure” repre 
sentatives of each of the three subfields 
were selected. Table 17 shows a compari- 
son of the three samples on selected varia 
bles which described their status in 1957. 

In an effort to isolate variables which 
would differentiate therapists, academicians, 
and administrators ( with the hope of there 
by being better able to understand the basis 
for their intravocational choices), Ss from 
the three subfields were compared on all of 
the variables from the measures selected for 
this study. Variables which differentiated 
between any two of the three groups at the 
level of statistical significance (chi 
square, corrected for continuity) were re 
tained for further analyses. 

All of the variables which differentiated 
any pair of the subfields were intercorre 
lated. Since many of the variables were 
highly correlated, a series of factor analyses 
was carried out to identify the underlying 
factors which could account for these rela 
tionships. The centroid solutions were ro 
tated by the Varimax method ( Kaiser, 
1958) and factor estimate scores were com 
puted in order to permit comparison of the 
three groups on each factor 


he Factors 
Specialization 


l’nderlying Pro 


Forty-eight variables were found to dif 


ferentiate significantly among therapists 


TABLE 17 
A COMPARISON OF THE THREE SPECIALIZATION 
SAMPLES ON SOME VARIABLES INDICATING 
THEIR STATUS IN 1957 


Assessment year: 
1947 
1948 


Present age: 
Mean age 
Range 


Geographical 
distribution: 
New York State 
New England and 
East Coast 

Midwest 

South 

Southwest and 
Rocky Mountain 
States 

West Coast 


Present field in 
psychology : 
Clinical psychology 
Experimental or 
social psy- 
chology 


Work setting: 

VA installations 

Colleges ot 
universities 

(sovernment 
agencies (non 
VA) 

Clinics (non 
hospital) 

Private practice 

Hospitals (non-VA) 


Satisfaction with pro- 
fessional choice 
(‘If I had my 
life to live over 
again, I would 
choose"’) 

Clinical psychology 
again 

Some other field in 
psychology 

Medicine 
(psychiatry 

Law or business 

Other 


Number of years past 
PhD (to 1957): 
Mean 


years 


number of 


Range 


Thera- 
pists 


20 


Acade- Admin- 
micians 


(N=20)| (N =20) 


34.7 
31-44 


wv 
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istrators 


15 


35.0 
31-41 


0 


20 


te 


10 


19 
| 4 | 
10 12 5 
32-44 
4 5 
2 8 8 : 
1 1 4 
2 0 1 ee 
= 14 = 
0 6 0 4 
4 
0 14 | 
0 2 
8 0 0 A : 
| 8 0 0 
0 0 3 
16 11 13 ; 
0 7 2 
4 0 
0 0 
0 2 
: — i $.7 
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academicians, and administrators, and nine 
factors appeared to underlie these variables. 
The nine rotated factors were named and 
interpreted as follows: 


Factor I: Childhood rebelliousness.* This 
factor appeared to tap the amount of con- 
flict and disturbance within the family 
group. On one pole of this factor were 
loaded variables such as reported friction 
with the father, discipline authority always 
one parent (as compared to shared respon- 
sibility for discipline), and childhood feel- 
ings of inferiority ; while on the other pole 
were such variables as reported understand- 
ing and permissiveness of the parents 
during the S’s adolescence, and over-all child- 
hood happiness. This factor markedly dif- 
ferentiated academicians (rebellious) from 
administrators (nonrebellious), and _ this 
difference can be interpreted in terms of the 
differing amounts of freedom from author- 
ity interventien which characterizes the 
work of the academician and the adminis- 
trator. 

Factor Il: Intrapsychic conflict. This 
factor was our nearest approximation to the 
“neuroticism” factor often uncovered in 
factor analytic studies. Highly loaded on 
one pole of this factor were such variables 
as reported childhood guilt feelings, vivid 
and frequent childhood fantasy, cutting 
classes frequently in college, eldest child in 
birth order, and a number of vocational 
preferences interpreted in terms of an un- 
derlying dimension of creative, yet neurotic, 
fantasy life. This factor differentiated acad- 
emicians (high conflict) from administra- 


’ The first six factors include variables from the 
BI. It is important to realize that these variables, 
based on the S’s evaluation of his childhood state 
while looking back at it from the vantage point of 
young adulthood, make any exact interpretation of 
their meaning hazardous because of their extreme 
subjectivity. For these variables the question can be 
raised as to whether differences in responses reflect 
genuine differences perceived in childhood, differ- 
ences perceived only when looking back upon child 
hood, or merely differences in the “honesty” of the 
respondent to admit socially unattractive person- 
ality attributes (response set). Consequently, al- 
though the factors are interpreted here in terms of 
actual developmental differences, the reader is cau- 
tioned that other interpretations are also plausible 
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tors (low conflict ), and this can be interpreted 
in terms of the differences in the use of 
fantasy in academic work as opposed to the 
reality orientation of administrative work. 


Factor III: Social conventionality. This 
factor appeared to tap a constellation of 
variables which may be thought of as char- 
acterizing “the all-American boy.” Highly 
leaded on one pole of this factor were such 
variables as Protestant or Catholic religious 
backgrounds (as opposed to Jewish fam- 
ilies, mixed marriages, or agnostic parents), 
regular church attendance, participation in 
college athletics, and political values (A-V). 
On the other pole were such variables as 
theoretical values (A-V) and high MAT 
scores. This factor—which can be interpreted 
in terms of the conventionality of the basic 
values being tapped—differentiated admin- 
istrators (highly conventional) from both 
other groups. Such a finding seemed con- 
gruent with a view of administrative activ- 
ities as most approximating the work of 
business personnel—who, in turn, might be 
expected to reflect the most conventional 
values of our society. 

Factor IV: Curiosity. This factor ap- 
peared to tap either some aspect of curiosity 
about the world or, perhaps, a particular 
kind of intellectual functioning. Loaded on 
one pole were variables such as difficulty 
with mathematics and science in high school 
and sex information received late in life; 
MAT score loaded on the other pole. The 
factor differentiated academicians (most 
curious) from administrators (least curi- 
ous) and seemed congruent with the differ- 
ing opportunities for satisfying curiosity in 
the two kinds of work. On the other hand, 
the reason why therapists failed to present 
a more homogeneous appearance on this 
factor was not understood. 

Factor I’: Exploitativeness. This factor, 
similar in many ways to factors found in 
other studies, was characterized by prefer- 
ences for leadership and administrative posi- 
tions and by high political and economic 
values (A-V). The term “go-getter” seems 
descriptive of individuals at one pole of this 
factor. Administrators were markedly more 
exploitative than either of the other two 


q 


groups, and this finding can be interpreted in 
terms of the greater opportunities for lead- 
ership inherent in administrative work. 

Factor VI: Vocational indecisiveness. 
This factor was difficult to interpret since 
few variables had high loadings on it. On 
one pole were such variables as deciding to 
become a psychologist late in college and 
preferences for leadership positions and for 
diagnostic interviewing; on the other pole 
was participation in college athletics. Ad- 
ministrators (least vocationally decisive) 
were differentiated from both academicians 
and therapists. 


Factor VII: Scientific values and inter- 
ests. This factor differentiated Ss whose 
Strong interest patterns approximated those 
of scientists in many fields (physics, chem- 
istry, engineering, mathematics, and experi- 
mental psychology) from Ss whose inter- 
ests more closely resembled sales personnel 
(sales managers, real estate and life insur- 
ance salesmen). Interestingly, reported dif- 
ficulty with mathematics and science in high 
school loaded highly on the sales pole of 
this factor, indicating an interrelation on 
this factor between abilities and interests. 
This factor markedly differentiated acade- 
micians (highly scientific) from administra- 
tors, a finding congruent with their later 
choice of professional specialization. 

Factor VIII: Creativity. This factor dif- 
ferentiated Ss whose Strong interest pat- 
terns resembled persons in artistic fields 
(artist, architects, author-journalists) from 
Ss more interested in business detail. 
Highly loaded on the artistic pole of this 
factor were theoretical values (A-V) and 
MAT score. Academicians appeared mark- 
edly more creative in interests than did ad 
ministrators, and these values appear con- 
sonant with the creative character of much 
of the work in the academic world. 

Factor 1X: Altruism. The social service 
scales on the SVIB defined this factor, 
which was expected to differentiate thera- 
pists from both other groups. Although 
therapists were slightly more similar in in- 
terests to persons in other social service 
fields than were academicians or adminis- 


trators, the differences were great. 


not 
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Moreover, therapists did not seem at all 
homogeneous on this factor. 

Table 18 contrasts the three groups on all 
nine factors. A rotated factor matrix show- 
ing the loadings of the significant variables 
on all factors can be found in Appendix C. 


TABLE 18 


INTERGROUP MEAN DIFFERENCES BETWEEN 
THERAPISTS, ACADEMICIANS, AND ADMINISTRATORS 
ON THE NINE FAcTors* 


Specialist Groups 


Factors 
Acade- | Thera- | Admin- 
| micians pists istrators 
Childhood 
rebelliousness high® med.» | 
Intrapsychic conflict high med. | low 
Social conventionality | low low high 
Curiosity high med. low 
Exploitativeness low low high 
Vocational 
indecisiveness low low high 
Scientific (Strong) high low low 
Creativity (Strong) high med. low 
Altruism (Strong) low- high- med. 
med. med. 


* Group differences are expressed as high, medium, or low 
on each factor. 


> All high vs. low differences significant: P < .01. All high 
vs. med. and med. vs. low differences significant: P < .0S 
Low-med. vs. high-med. difference significant: P < .0S. All 
other differences nonsignificant (Duncan's Multiple Range 
Test). 


Personality Descriptions of the Modal 
Therapist, Academician, and Administrator 


The following is a summary description 
of tne three subfields in terms of the varia- 
bles which differentiated them. All state- 
ments are to be regarded as relative only 
to the three groups under comparison—and 
not necessarily to “people in general.” 

Academicians. The combined academi- 
cian sample turned out to present a reija- 
tively homogeneous personality picture—in 
spite of the fact that it contained a number 
of individuals who were presently not 
working in university settings. It would 
thus appear that our conceptualization of 
an academically oriented group within the 
larger professional field was a useful for- 
mulation. 
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The early childhood of the academician 
would appear to be characterized by a con- 
siderable amount of intrapsychic disturb- 
ance, perhaps brought about in part by a 
rather dominant mother (with whom the 
child was enveloped in ambivalent friction) 
and a more taciturn easy-going father. At 
any rate, academicians reported having had 
considerable guilt feelings as children, a 
good deal of flight into fantasy, and more 
than average guilt reactions to masturba- 
tion. They were likely to experience feel- 
ings of inferiority and in general to view 
themselves as somewhat unhappy during 
childhood. Moreover, they were not likely 
to be leaders of their peers and their rela 
tionships with other children may well have 
caused them some anguish. Not being ath- 
letically inclined, they apparently did not fit 
easily into the group activities of childhood. 

On the other hand, their relative isola 
tion and solitude as youths—combined with 
their need to rely on their own fantasy and 
creative imagination for stimulation—ap 
pears to coincide with their later choice of 
work. The working activities of an acade 
mician can be viewed as relatively solitary 
and creative, in contrast to the interpersonal 
and organizational activities of many other 
fields. The introspective orientations of 
academicians as children seemed to have 
paved the way for continued activities of 
this nature. 

Moreover, the old observation that less 
popular children often compensate for their 
lack of peer attention by turning their ener- 
gies to scholastic pursuits seems to hold 
here. Academicians appeared to have been 
curious as youths and to have done well in 
school. They were probably more serious 
students throughout their schooling, and 
they decided on a specific professional ca- 
reer early in their college lives. By this 
time they had rejected many of the conven 
tional values of society in favor of ones 
more suitable to their introspective orienta 
tions. 

One can see the prospective academician 
in college as bright, theoretically oriented, 
widely read, and not socially active. He has 
rejected religion—at least in its conven 
tional forms—and he is neither exploitative 


nor pecuniary. He sees the world as a rel- 
atively unkind place and is determined to 
do something about it. More importantly, 
perhaps, he has found an outlet in books 
and research for many impulses which he 
is not adept at expressing interpersonally. 

Upon entering graduate school, the pros- 
pective academician is still likely to be a 
serious student and to be more concerned 
with ideas themselves than with utilizing 
them to manipulate other persons. His in- 
terests approximate more nearly those of 
scientists—perhaps in all fields—and artistic 
persons in many areas. His 
values are diametrically opposed, in many 
ways, to individuals in the business world. 
In a word, he is not the “organization 
man. 


or creative 


Administrators. Since administrators and 
academicians in the field of clinical psychol- 
ogy both chose the same profession in 
which to spend their lives, they must have 
shared a certain commonality of early de 
velopmental experiences and later interests 
and values by the time they entered grad- 
uate school. Nevertheless, on many dimen- 
sions, administrators appeared considerably 
different. They, too, formed a remarkably 
homogeneous group, in spite of the fact 
that they differed with respect to the kinds 
of settings in which they performed their 
administrative functions. 

If the academician could be characterized 
as an introvert, the administrator emerged 
as an extrovert. Often the youngest or only 
child from a family where the parents 
shared the same Protestant or Catholic 
faith, the prospective administrator saw the 
authority for his discipline shared by both 
his mother and his father and experienced 
little friction with either parent. Of all our 
groups, his was the happiest childhood—or 
so it would appear from what he reported 
of it. He was least likely to report recourse 
to vivid or frequent fantasy; he was least 
often plagued with feeling of guilt or in 
feriority; he perceived his parents as most 
understanding and permissive. 


Perhaps even more significant, his rela- 
tionships with his peers were reported as 
considerably more pleasant. He was likely 
to have been a member of a childhood gang, 


and as often as not was its leader. In high 
school and college he participated in ath- 
letics and was often elected to class and 
club offices. He was more likely to have 
some trouble with mathematics and science 
and was probably not as serious a student, 
in general, as the prospective academician. 
Perhaps he was less curious—he learned 
about sex a little later in life—and was 
more interested in social activities. 

In college his interests and values were 
more conventional ones, shared—one might 
guess—by most of his college peers. He was 
religious, in the sense that he attended 
church more regularly and considered him- 
self a Protestant or Catholic. He decided 
on his professional career later in college, 
perhaps only after some thought of entering 
the business world. In general, though he 
was not a scholar, he attended classes faith- 
fully (like church, perhaps), and no doubt 
achieved fairly high grades. On the other 
hand, he scored lower on the MAT and was 
less likely to turn in an outstanding aca- 
demic performance. 

As he entered graduate school, when the 
administrator thought of future specializa- 
tion he was more likely to prefer diagnostic 
interviewing (surely mot marital counsel- 
ing). More than academicians or therapists, 
he wanted to be a leader in the field, a de- 
partmental chairman, or a director of some 
project. The interpersonal orientation so 
characteristic of him during his develop- 
mental years was carried over to his choice 
of specialty: he would still be a “member 
of the gang,” still be a “leader.” 

Since every person shares many of the 
same human attributes, the above descrip- 
tions of academicians and administrators can 
best be described as caricatures of the dif- 
ferences that were found to exist between 
them. Obviously, most of the individuals in 
each group would not fit these abstract 
molds perfectly. However, one can estimate 
the extent to which these differences, taken 
in combined form, actually did differentiate 
the academician from the administrator 
samples by combining the factor estimate 
scores over the eight factors which indi- 
vidually differentiated the two groups. 
When this was carried out, the degree of 
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overlap between academicians and adminis- 
trators was found to be minimal. Splitting 
the pooled scale at the median score, only 
two academicians overlapped on the admin- 
istrator half of the distribution, while no 
administrators were misclassified. Thera- 
pists, on the other hand, appeared to be nor- 
mally distributed on this combined dimen- 
sion. 

Therapists. In the original selection of 
Ss for this study, the therapist group ap- 
peared to be the purest sample of persons 
representing any of the areas of specializa- 
tion; consequently, we expected them to 
present a more homogeneous appearance 
on the variables considered in the study 
than either of the other two subfields. In 
fact, quite the contrary occurred! 

This failure to identify more adequately 
the unique characteristics of therapists pre- 
sents one of the most enigmatic findings of 
this study. The working activities of thera- 
pists—regardless of their therapeutic orien- 
tations—-seems so similar that one might 
expect their occupational choices to be 
equally similarly motivated and their de 
velopmental backgrounds to present a rela- 
tively homogeneous appearance. Our find- 
ings indicate this is not the case. 

Our picture of the therapist, as it 
emerges from those few variables which 
discriminated the group from the others, is 
spotty and uneven—without the consisten- 
cies we found in the case of the academi- 
cians and the administrators. The therapist 
was more likely the eldest child in a Jewish 
family. He reported considerable friction 
with his father and professed a deep attach- 
ment to his mother, suggesting possible fail- 
ure to resolve the theoretical Oedipus con- 
flict. If we accept this type of formulation 
of the therapist's dynamics, it might explain 
his reported greater involvement in vivid 
and frequent fantasy. As a child, the thera- 
pist was likely to experience feelings of in- 
feriority and to share with the academician 
a disinclination toward athletics. 

In high school, although his values are 


® Obviously, a replication is necessary in order to 
iscertain the loss in discrimination resulting from 
e removal of chance occurrences 


‘ 
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likely to have been theoretical ones, he is 
likely to have had some difficulty with 
mathematics and science. By the time he 
entered graduate school he had already ten- 
tatively decided on doing therapy and was 
especially intrigued with marital counseling ; 
being the inventor of a personality test or 
the author of a psychological theory also 
had some appeal. In interests, he resembled 
personnel directors, public administrators, 
and—to a lesser extent—pharmacists; he 
appeared relatively dissimilar to architects. 
His interests were rather close to those of 
guidance and industrial psychologists. 

Probably as significant as these variables 
which discriminated therapists from the 
other groups was the fact that most of the 
variables considered in this study failed to 
differentiate therapists at all. In general, 
therapists can be found evenly scattered 
throughout the entire range of most of the 
variables in our analyses, indicating that 
this group is an extremely heterogeneous 
one. 


A Small Replication 


In view of the failure to find replications 
of significant relationships between predic- 
tor and criterion variables (as reported 
earlier) it seemed appropriate to question 
the degree of generality which may be at- 
tributed to the findings reported here. Since 
the specialist groups were selected as the 
“purest” cases from the combined 1947 and 
1948 samples, it was not possible to repli- 
cate these analyses. However, as a check 
on the probable generality of the findings, 
a test was made of their applicability to 10 
new Ss selected to approximate the criteria 
of professional specialization used ; in com- 
parison with the 60 Ss on whom this study 
was based, they were less “pure” represent- 
atives of the three areas of specialization 
within clinical psychology. Three therapists, 
four academicians, and three administrators 
composed this new group. 


Factor estimate scores were computed for 
each of these 10 new Ss on the basis of 
their test scores and responses to the BI, 
and then these factor scores were pooled 
to get estimates of the similarity of these 10 


Ss to the characteristics of the previous 
therapists, academicians, and administrators. 

The resultant findings were very encour- 
aging. All four of the persons in the new 
sample of academicians achieved factor 
scores well into the range of the original 
academicians; moreover, two of the three 
administrators achieved factor scores in the 
range of the original administrator sample. 
If we were to have predicted the specializa- 
tion of these seven Ss, we would only have 
misclassified one (and his combined factor 
score was quite close to the median score of 
the original 60 Ss). The three therapists, 
as expected, were less homogeneous; how- 
ever, two of the three achieved scores far 
into the therapist portion of the combined 
scale which differentiated the original 20 
therapists. These findings cannot be con- 
sidered as a replication of the original 
study, but the congruence of the personality 
characteristics of 10 new Ss to the original 
sample certainly suggests the possibility of 
wider generality of these findings. 


SUMMARY 


lwo hundred and forty-five Ss, assessed 
in the VA Selection Research Project in 
1947 and 1948 just prior to their entry into 
graduate school in psychology, were fol- 
lowed up by a mail questionnaire in 1957. 
Responses to the questionnaire yielded in- 
formation concerning their occupational and 
educational histories since graduate school, 
their present areas of specialization within 
the profession, and their present satisfac- 
tion with their previous vocational choices 
and their choices regarding specialization. 
An index of their scholarly productivity 
was obtained by a search of Psychological 
Abstracts. 

Approximately 70% of the original Ss 
had achieved a PhD in psychology by 1957 
while 5% still considered themselves gradu- 
ate students; the modal number of years 
taken to obtain a PhD was five. Roughly 
50° of the total sample stated that they 
would choose some vocation other than 
clinical psychology, if they could live their 
lives over again, while approximately 40% 
of those Ss who had achieved the PhD in 


psychology expressed this form of dissatis- 
faction with their previous professional 
choice. Satisfaction appeared to be related 
only in small part—to present work setting 
and present area of specialization within 
psychology. The scholarly productivity of 
the total sample seemed unimpressive, with 
nearly one-half of the Ss having no publi- 
cations in psychology and another quarter 
having only one such publication (typically 
based on a PhD dissertation) during an 
eight-year period. 

Nearly 200 of the earlier assessment var- 
iables, mostly from objective tests and a 
Biographical Inventory, were correlated 
with criterion measures of academic attain- 
ment, scholarly productivity, American Psy- 
chological Association affiliation, present 
identification with clinical psychology, satis- 
faction with previous professional choice, 
satisfaction with choice of specialization, 
and a combined criterion of academic suc- 
cess and simultaneous satisfaction with pro- 
fessional choice. general, predictor- 
criterion intercorrelations were low, in no 
case accounting for more than 10% of the 
criterion variance. Moreover, for each cri- 
terion a set of significant correlations for 
the 1947 sample disappeared upon replica 
tion with the 1948 sample. Possible expla 
nations for these findings were discussed. 
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\. Regarding the Subjects and Their Training 

1. Graduate students in psychology participating 
in the VA program are remarkably similar to other 
graduate students in psychology not majoring in the 
clinical area. Both are highly selected groups of 
students, not only in intelligence but also in terms 
of their measured social and emotional adjustment 
Both show a pattern of interests and values charac 
teristic of professional and scientific persons. The 
VA students differ slightly from the others primar- 
ily on the basis of interests more characteristic of 
the professions which involve contact with people, 
social welfare, and persuasive activities; the inter- 
ests of the non-clinical students are somewhat 
more similar to professions which are primarily 
concerned with things and ideas. 

2. Groups of trainees enrolled at different train 
ing institutions show significant differences on 
measures of ability and achievement. 

3. Trainees report wide differences in emphases 
of training programs at different universities and 
in their clinical experiences in field training 

4. Ninety-five of the 128 P-1 trainees assessed 
and entering the four-year program in 1947 will 
probably complete their training in the VA _ pro- 
gram. For this group, then, the total attrition is 
ibout 20%. However, at least seven more of the 
original group are known to be completing their 
PhDs in psychology, some with clinical specializa 
tion but not as VA trainees. 

5. About one-third of the trainees studied report 
having received some personal therapy during their 
training. 

6. The Strong Vocational Interest patterns ot 
young clinical psychologists were found to be much 
more similar to those of young psychiatrists than 
to those of young physicians. The psychiatrists and 
clinical psychologists were much more similar in 
pattern of interests than were psychiatrists and 

ther physicians, 
3. Regarding Criteria of Professional Success 

1. There is no satisfactory single criterion of 
success in training for or practicing clinical psy- 
chology. University staff members and supervising 
clinical psychologists show wide individual differ 
ences in their conceptions of successful academic 
performance and of professional skills 

2. There are three general components of success 
in clinical psychology: (a) intellectual accomplis! 
ments (e.g. academic achievement and research 
productivity), (>) the clinical skills of diagnosis 
ind therapy, and (c) skills in social relations. In 
general, judges agree much better on the first than 
on the latter two components 

* This section is quoted directly from Kelly 
Fiske (1951, pp. 193-197) 
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MAJOR FINDINGS OF THE ORIGINAL KELLY AND FISKE STUDY* 


3. Supervisors’ ratings do not adequately differen- 
trate the various job functions of the clinical psy- 
chologists 

+. Ratings of clinical competences appear to be 
of the role of the rater (e.g., 
colleague) as of the person 


as much a function 
teacher, supervisor, 
being rated. 

5. No completely acceptable objective measures 
f competence in the purported functions of the 
clinical psychologist are available, but certain 
promising techniques for evaluating clinical skills 
were developed within this project. However, fur- 
ther developmental work on these measures is nec 
essary before they can be used in the routine evalu- 
ition of clinical competence. 

In summary, we may say that there are several 
ind alternative criteria of success in clinical 
psychology. However, the rated criteria showed 
sufficiently high intercorrelations to identify a com 
mon core of what might be called “perceived clin 


1 competence.” 


iried 


the Prediction of Success in Clini 
il Psychology (for the 1947 subjects) 
\ccuracy of prediction of success in clinical 
hology is a function of the job component be 
ng | redicted 
>. Th 


raining 


e intellectual aspects of success in graduate 
ind also the general aspect of over-all 

linical competence can be predicted surprisingly 
well in view of the fact that the group of students 
‘ntering training is already highly selected. Valid- 
criteria range from .35 to .60 even 
when the range of talent is further restricted to 
those completing this doctoral program. Further- 
re, the validities are based on a national sample; 
known inter-institution and inter-installation differ- 
es in caliber of students and emphases of train- 


ties for these 


ing presumably lower the obtained relationships 

3. Prediction of relative standing within the 
ompleting professional training is generally 
irate than the prediction of outcome of 
training (Success/Failure). This is probably due 
to (a) the diversity of causes for leaving the pro 
t academic failure, health, preference for 
other fields, etc., and to (5) differences in the cri- 
teria applied by various universities for continuing 


nrooeroar 


more cc 
e 


clinical aspects of professional 
. less tangible, and therefore not 
s readily evaluated, are less predictable than the 


The more 


wwetence. being 


intellectual component of professional success 
Fach of several criteria of diagnostic and thera- 
peutic competence is predicted by one or more pre- 
lictors, but different criteria of these skills tend 
» be predicted by different predictors 

5. Those aspects of professional success involv 


wel «htt 
skills 


ire also predictable bevond chanx 


but because measures of these skills are in part a 
function of the persons evaluating them, they tend 
to be less predictable than intellectual aspects of 
professional success 

6. For the group of trainees who entered train 
ing in 1948, the validities of both objective test 
scores and assessment ratings based on a modified 
assessment program were found to be much lower 
than for the group entering in 1947. These lower 
validities may be a function of any one or any 
combination of the following factors: 

(a) Test data and other materials were obtained 
for the 1948 group before action was taken on thei 
applications for admission to graduate school 

(b) The younger, intellectual!) 
more able, and more homogeneous with respect to 
both age and intellectual ability 


1948 group is 


(c) The 1948 group had been in training only 
two and one-half vears at the time criterion meas 
ires were collected 
) Regarding the Relative Efficienc Various 


Predictors 


The potential validity of all predictors is ap 
parently limited by inadequacies in available cri 
terion measures. All criterion measures used and 
-onsidered are more or less fallible because of on 
1x more of the following characteristics. They 
may (a) have low reliability; (b) be too general 
yr too specific; (c) be lacking in validity, 1.e., not 
relevant to actual competence; (d) be too provin 
ial, ie., reflect only the degree to which profes 
sional behavior conforms to that 
subculture 


expected in 


Each of the several criterion measures 


predicted about equally weil by any one of several 


techniques or procedures, This is especially so 
the criteria of intellectual performan eg, rated 
Academic Performance, scores on Content Exami 


nation and rated Research Competence) 


3. In general, only a small proportion of the ob 
tive test scores were found to correlate signif- 
antly with the criterion measures 
two from. standard 
predict most of the criterion about as 
well as the best of the assessment ratings by profes 


However, one 
scores psychol ywical tests 


measures 


sional staff members. Although assessment ratings 
ind objective test scores were found to be about 
«qual in predictive efficiency, objective tests are, 
wurse, much more economical in terms of time 
nd cost. The most generally useful of the obje 
tive tests were the Miller Analogies Test and the 
Strong Vocational Interest Blank. Scores on on 
r both of these tests predicted most of the se 
ral criterion measures with fair efficier 


4. With respect to the descriptive, evaluative an 
nredictive ratings made by staft 
hers, it was found that: (a) predic tive rati 
‘air to high interjudge reliability } 
the same materials; 


assessment mem 


s show 


when based or 
(b) the relationship betwee: 
prediction and the 


which the predictions 


the 


accuracy of assessment 


ount of materials on 
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hetween the final st 


three and one-half vears later 


based is markedly non-linear. In general, assess- 
ment predictions based on the credential file plus 
the objective test profile tend to be almost as accu- 
rate as those based on more materials including an 
\utobiography, projective tests, interviews, and sit 
uation tests; (c) the findings do not reveal any 
‘lear-cut superiority in the validity of pooled rat- 
ings made in a “staff conference” or of arithmeti- 
-al ratings over those of the individual staff mem- 
bers; (d) self-ratings by the candidates have some 
predictive value for criteria of intellectual success 
but show no relationship to other criteria of pro- 
fessional success; (¢) judgments made by assess- 
ment teammates and judgments by staff members 
on the basis of observing the candidates only in 
situation tests both have some predictive value; in 
general they are about as valid as staff ratings 
based on the credential file alone; (f) predictions 
»f success in clinical psychology made on the basis 
»f single projective techniques tend to have very 
low correlations with any of the criterion meas 
ires. The same is true for predictions based on an 
integrative study of the protocols for the four pro 
jective techniques 

5. Predictions of success in the training program 
made by university staff members on the basis of 
procedures conventionally used by departments in 
he selection of students tended to correlate as well 
with criteria of academic performance and research 
ompetence as any assessment predictions. This was 
ad not been seen 
the time of mak- 
generality of this 
by the fact that university staff 
predictions were available for only half of the sub 


rue even where the candidate h 
by the university staff member at 
ie predictions. However, the 
finding is limited 


jects, and criterion measures for this subgroup 
were more predict ble than those for the total 
oup of subjects 
Other Findings 
Qualitative assessment evaluations for 20 


were clinically 


nees 


matched with qualitative 


valuations of on-the-job functior made three 
nd one-half years later with an accuracy of 72% 
where the chance value was 50%. The median phi 


~oefficient 


for individual cases was .44. The same 
were matcl 


hed objectively, on the 


ases 


basis of 
the profiles of personality ratings by teammates at 


the time of and profiles of ratings made 


assessment 
on the same variables by colleagues three and one 
half accuracy of 74%. The 
median phi coefficient was .60. Both of these 
matchings are significantly better than chance 


years later, with an 


> Ratings on all personality variables by staff 
ind teammates at the time of assessment tend to 


w low correlations with ratings on the same 
iriables made colleagues three and one-half 
ears later. These correlations are lower than those 


aff predictive ratings at the 


me of asses 


‘nt and criterion me 


sures obtained 
Whether these data 


»f consistency of personality among 


1 


| 
| 
re indicate a 
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the subjects over the period of graduate study re- 
flect different frames of reference for different 
raters, or merely reflect the relative atypicality of 
the behavior displayed by candidates at the assess- 
ment, is not known. 

3. None of the rated criterion measures of clini- 
cal competence or scores on the objective Content 
Examination were found to differentiate the skills 
of trainees with three and one-half as compared 
with two and one-half vears of training. This find- 
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ing may be the result of the latter group having 
been more highly selected or better trained. How- 
ever, a number of lines of converging evidence 
lead us to question whether present advanced train- 
ing procedures, or additional years of experience 
lead to any currently measurable improvement in 
linical skills. This may be a function of the na- 
ture of the training and/or experience, or it may 


reflect the inadequacy of available criterion 


merely 
measures of clinical skills 
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APPENDIX B 


Veterans Administration Selection Research Project, 1027 E. Huron Street, 
Ann Arbor, Michigan 


1957 Preliminary Follow-up Questionnaire 
Code number__ 
(In order to preserve complete anonymity, please do not sign your name on this form.) 
1. Graduate school(s) attended, dates of entry and leaving, and degree(s) received: 


Schoo! Dates Degree(s) 


Field of graduate specialization :_ 
(if other than ciinical psychology ) 


Reason for change of field from clinical psychology (if applicable) 


Reason 


Please list the positions you have held during the past seven years, starting at the present and work 
ing backward in time. List any part-time work performed since leaving graduate school (including 
private practice, if you had one). 


IN THE CASE OF WORK RELATING TO THE FIELD OF PSYCHOLOGY, please try to 
estimate—as accurately as possible—the percentage of time you spent in each of the following roles 
Administration or supervision, diagnosis, research, teaching, and therapy. In cases where your job 
function changed over the years on one job, please indicate such shifts below 

IN THE CASE OF PERSONS WORKING OUTSIDE THE FIELD OF PSYCHOLOGY, please 
cross out the listed job functions and substitute ones more appropriate to your work. 


Percentage of time spent in 
Title, or Name 
of Position Emplover Location Dates Adminis 


(e.g.. Staff Psychologist, A. Hospital, Downey, Ill. °52-55) tratior agnos tesearc! Teaching Therapy 
Super % 


© 


20 
| 
| 
: 

4. [EE for leaving graduate school prior to Ph.D. (if applicable) :— mee ca 
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(The answers to questions on this page are requested when they are applicable.) 


6. At the present time, I consider myself primarily a: ___Therapist 
(Rank order your answers from 1 (MOST) to 5 (LEAST) ; ___Teacher 
indicate ties by duplicate ranks.) Researcher 


_Diagnostician 
‘__Administrator- 
Supervisor 


If it were not for considerations of training, status, salary, etc. Therapist 


(i.e., in terms of the actual work invelved), I would prefer working Teacher 
asa: Researcher 
(Rank as above) Diagnostician 


\dministrator- 
Supervisor 


In the years ahead, taking everything into consideration, the odds Therapy 
are that I will devote most of mv time to -Teaching 
(Rank as above) -Research 
Diagnosis 

_Administration 

Supervision 


If I had my life to live over again (knowing what I now know), I would try to end up in 
(check one) 


___clinical psychology 
_some other field in psychology 
_law or business 
_medicine 

other (please specify) :— 


10. In the years ahead, I intend to work in the following kind(s) of setting: (check one or more) 


_My home or private office 
_A clinic 
An N.P. hospital 
_A general medical hospital 
A University, in the Dept. of Psychology 
___A University, in a clinic or counseling offic 
_Business or industry 
A government agency; specify: 
Other; specify: - 


Thank You ! 
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Variable 


Reported severity of inferiority 

feelings in childhood (BI) 

Reported amount of friction 
with father (BI) 

Discipline authority always one 
parent—rather than shared 
(BI) 

Reported amount of friction 
with mother (BI) 

Perceived permissiveness of 

parents during S’s adolescence 


(BI) 


Reported degree of childhood 


happiness (BI) 
Objectivity (G-M) 


Reported frequency of cutting | 


classes in college (BI) 
Reported amount of guilt reac- 
tions to masturbation (BI) 


Reported amount of guilt feel- | 


ings in childhood (BI) 


Preference for marital counsel- | 


ing—as opposed to other 
forms (BI) 

Reported frequency of fantasy 
in childhood (BI) 


Feel authors best understand | 


human nature (BI) 
MAT 
Birth order: eldest child (BI) 
Preference for leadership activi- 
ties—I (BI) 
Reported regularity of church 
attendance (BI) 
Religious values (A-V) 
Reported amount of participa- 
tion in college athletics (BI) 
Political values (A-V) 
Religion—Protestant or Catho- 
lic vs. Jewish or none (BI) 
Parents’ religion—Protestant or 


Catholic vs. Jewish, mixed, or | 


none (BI) 

Theoretical values (A-V) 

Age when sex information re- 
ceived (BI) 

Reported amount of difficulty 
with math in high school (BI) 

If can’t meet deadline, tell 
superior (BI) 

Economic values (A-V) 

Preference for leadership activi- 


ties—II (BI) 


46 


46 


30 


IN 


Rotated Factors 


33 
30 
30 
38 
32 
38 37 
26 
71 
70 
52 .50 
49 
71 
37 
53 25 
40 
— .38 
58 


FACTOR MATRIX* 


PSYCHOLOGY 


. 26 


49 


53 


31 | 
; 
I I IV \ VI VIL | VIE | IX 
53 
.58 
51 
.50 
42 
.33 .40 
31 39 
32 
— .42 28 
28 
— .30 - | - | a 
-. 
! 
53 50 
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Variable 


Experimental psychologist 
(SVIB) 

Real estate salesman ((SVIB) 

First decided to become a psy- 
chologist early vs. late in life 
(BI) 

Preference for diagnostic inter- 
viewing—as opposed to ther- 
apy or research (BI) 

Physicist (SVIB) 

Chemist (SVIB) 

Engineer (SVIB) 

Mathematician (SVIB) 

Life insurance salesman (SVIB) 

Sales manager (SVIB) 

Guidance psychologist (SVIB) 

Artist (SVIB) 

Architect (SVIB) 

Author-journalist (SVIB) 

Office manager (SVIB) 

Industrial psychologist (SVIB) 

Purchasing agent (SVIB) 

Social worker (SVIB) 

Public administrator (SVIB 

Social science high school teach- 
er (SVIB) 

YMCA secretary (SVIB) 

Personnel director (SVIB 


® This table is a composite of a number of rotated factor matrices; the relation, centroid, transformation, and rotated 
matrices can be found in Goldberg (1958). 
» Only factor loadings of .25 or higher are listed 


32 
Rotated Factors 
I II Ill lV V VI VII VIII | IX 
.80 
31 - 85 
.58 
.35 
28 91 
90 
.90 - 
30 .85 — .45 
— .80 31 
83 
} .70 
.57 
-.28 —.78 | .34 
.35 — .67 
63 
38 | .78 
— .41 25 | .76 
.63 
50 
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